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ENHANCING  TECHNOLOGY  TRANSFER  OF  COMPUTER  HARDWARE  AND  SOFTWARE 
ARCHITECTURES  USING  HUMAN  FACTORS  IN  INITIAL  DESIGN 

BACKGROUND 

Technology  advances,  particularly  in  the  field  of  automation, 
have  far  outstripped  the  capacity  of  operational  users  to  recognize  and 
assimilate  all  the  possible  benefits.  The  research  community  has 
produced  a  large  variety  of  capabilities  which  may  be  categorized  as 
"decision  aids"  and/or  "operational  aids"  and  has  concurrently 
experienced  considerable  difficulty  in  effecting  technology  transfer, 
i.e.  acceptance  of  the  research  by  the  user  or  operational  community. 

In  October,  1977,  the  Naval  Postgraduate  School  (NPS)  in 
Monterey,  California,  established  a  new  curriculum  in  Command,  Control 
and  Communications  (C3).  This  curriculum,  as  described  in  the  Command, 
Control  and  Communications  (C3)  Program  section  of  the  Naval 
Postgraduate  School  Catalog  (1978),  includes  within  its  objectives,  a 
major  goal  "...  to  provide  the  students  with  enhanced  capabilities  to 
operate  effectively  in  such  diverse  areas  as  military  decision  making, 
current  and  future  C3  systems  design,  crisis  management,  ...  ".  Two 
courses  within  the  C3  curriculum,  OS  3871:  Man-machine  Interaction,  and 
CS3750:  C3  Exercise  Laboratory,  are  specifically  designed  to  introduce 
research  that  is  directed  toward  future  C3  systems.  As  such,  they 
represent  unique  vehicles  for  studying  the  relevance  of  research 
products  in  that  the  C3  student  body,  composed  of  professional  military 
officers,  experienced  in  operational  requirements,  is  exposed  to  the 
various  research  products  and  then  provided  the  class  time  and  equipment 
to  use,  experiment  with,  and  evaluate  each  research  product. 

The  C3  curriculum  has  received  support  from  the  Defense 
Advanced  Research  Projects  Agency  (DARPA),  a  Department  of  Defense  (DoD) 
agency  specifically  tasked  to  conduct  exploratory  research  in  advanced 
technology.  Much  of  the  research  in  command,  control  and  communications 
is  a  product  of  DARPA 's  Information  Processing  Technology  Office.  These 
research  products  fall  into  a  variety  of  areas  such  as  graphics 
languages  and  protocols,  distributed  data  base  concepts,  production  rule 
systems,  and  natural  language  query  systems  as  examples.  However,  it  is 
often  difficult  to  evaluate  research  products  like  these  from  an 


operator's  perspective  and  when  this  is  done,  the  product  is  often  so 
far  developed  that  it  is  infeasible  to  change  it  ...  once  again,  a 
reminder  of  the  need  to  conaider  human  factors  in  initial  design  stages 
of  any  computer  hardware  or  software  development. 

With  the  establishment  of  the  C3  curriculum  and  the  close 
association  with  DARPA,  a  unique  situation  had  developed  in  which 
military  officers  in  the  C3  curriculum  would  be  able  to  use  and  evaluate 
emerging  technologies  as  they  develop  and  provide  operational  user 
feedback  to  the  research  community.  To  enable  this  to  occur,  each 
officer  in  the  C3  curriculum  has  been  provided  with  a  portable,  hardcopy 
producing,  computer  terminal.  A  local  telephone  call  allows  the  officer 
to  connect  into  the  ARPANET,  a  distributed  network  of  over  100  different 
large  scale  computers.  The  geographically  separated  host  computers  are 
linked  together  by  a  communications  net  to  form  an  operational,  resource 
sharing  network  of  a  wide  variety  of  DARPA  sponsored  research  centers 
and  other  DoD  activities  in  the  continental  United  States,  Hawaii, 
Norway  and  England. 

Originally  implemented  in  1969  under  a  research  and  development 
program  sponsored  by  DARPA  (then  ARPA)  to  advance  state-of-the  art 
computer  internetting,  the  NET'S  use  has  experienced  growth  in  many 
unplanned  directions.  The  ARPANET'S  communication  system  uses  the 
technique  of  "packet  switching"  where  each  message  is  separated  into 
small  packets  and  each  packet  is  then  switched  to  its  destination  via 
the  fastest  communications  path  available  at  any  instant.  Upon  arrival 
at  the  destination,  the  packets  are  reassembled  in  the  proper  order. 

Since  DARPA 's  Command,  Control  and  Communications  research 
products  reside  on  host  computers  around  the  ARPANET,  the  officers  in 
the  UPS  C3  curriculum  can  therefore  acoess  these  research  technologies 
at  any  time.  For  example,  a  student  officer  can  work  with  a  given 
research  technology  at  a  computer  in  California  and  within  seconds  be 
working  on  a  real  time,  interactive  basis  with  a  different  research 
technology  which  may  reside  on  a  host  computer  in  Pennsylvania. 


OBJECTIVE 


The  authors  had  both  encountered  a  variety  of  computer  systems 
before  but  one  outstanding  feature  of  the  ARPANET  and  most  of  the 
Command,  Control  and  Communication  technologies  on  it  is  that  they  have 
been  designed  with  muoh  consideration  for  the  operational  user. 
(Finally  human  factors  consideration  in  the  design  stage!)  This  was  one 
of  the  primary  reasons  for  deciding  to  introduce  the  new  C3  class  to  the 
ARPANET  and  its  C3  technologies. 

Many  students  find  most  computer  systems  too  detailed  and 
exacting,  e.g.,  if  a  dollar  sign,  slash,  asterisk,  etc.,  is  not  entered 
in  exactly  the  proper  format,  the  Job  will  fail  and  the  students  become 
very  frustrated  and  "turned  off"  because  the  system  is  not  friendly. 
This  feeling  is  further  accentuated  in  military  officers,  especially 
those  in  the  C3  curriculum,  who  represent  the  Army,  Navy,  Air  Force  and 
Marines,  and  who  may  be  operational  users  of  computer  systems  but  who 
have  no  desire  to  worry  about  system  details  like  dollar  signs.  They 
want  to  be  able  to  get  their  operational  answers  from  the  computer 
system  as  quickly  as  possible,  and  then  get  baok  to  their  operational 
duties. 

It  was  therefore  the  objective  of  this  study  to  introduce  a 
group  of  relatively  naive  computer  users  to  a  computer  network  system 
(the  ARPANET  and  its  technologies)  which  was  very  user  oriented  in 
design.  The  question  to  be  addressed  was  would  an  educational  exposure 
to  such  a  system  change  the  students'  perceptions  and  attitudes  toward 
computer  systems,  and  if  so,  how? 


METHOD 


Subjects 

The  subjects  were  members  of  the  first  class  in  the  C3 
Curriculum  at  NPS.  All  branches  of  the  U.  S.  Armed  Services  were 
represented  with  ranks  ranging  from  a  Navy  Ensign  to  a  Marine  Corps  Lt. 


Colonel.  Prior  to  this  study,  each  student  had  just  completed  an 
introductory  computer  programming  (FORTRAN)  course  and  an  introductory 
computer  graphics  course,  typical  of  those  found  in  many  universities. 
In  those  oourses  they  had  done  the  usual  types  of  tasks  such  as  writing 
specific  programs,  setting  bits  to  adjust  vectors  on  cathode  ray  tubes 
(CRTs),  and  doing  term  projects.  In  general,  they  had  a  very  common 
computer  oriented  background.  Most  of  them  thought  what  they  had 
learned  so  far  was  "nice  to  know"  but  they  really  had  no  desire  to  work 
with  automated  systems  and  learn  more  of  the  nitty-gritty  details. 

Procedure 

The  subjects  were  all  given  a  questionnaire  prior  to  the 
beginning  of  the  experiment  and  prior  to  receiving  their  personal, 
portable,  oomputer  terminals  or  any  information  about  events  or  topics 
coming  in  the  future  related  to  this  experiment. 

Subjects  received  no  monetary  rewards  for  participation  in  the 
following  activities. 

Standard  background  information  about  operational  experience  and 
previous  computer  experience,  ages,  rank,  etc.,  were  obtained  from  the 
questionnaires.  In  addition,  30  questions  were  asked  beglnnning  with 
the  phrase  "To  What  Extent  ...  "with  responses  A  through  E  ranging  from 

A.  None,  or  very  little 
to 

E.  A  great  deal 

These  30  questions  tried  to  determine  the  subjects'  perception 
of  the  value  or  utility  of  automatic  data  processing  and  automated 
decision/operational  aids.  The  questions  covered  a  broad  area  and  were 
similar  to  the  following: 

"To  what  extent  do  you  feel  you  could  have  effectively 
used  a  computer  or  computer  network  in  your  past 
Jobs?" 
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Questionnaire  responses  were  received  from  all  subjects  and  not 
examined  until  the  end  of  the  experiment. 

The  subjects  were  each  then  given  a  portable,  hard-copy, 
computer  terminal  to  take  home  in  addition  to  a  srall  introductory 
pamphlet  (Holg,  1977)  explaining  the  ARPANET  in  general,  and  one  of  itr 
message  handling  systems.  They  were  told  a  class  would  begin  in  four 
days  in  which  they  would  make  use  of  the  terminals  on  the  ARPANET. 

The  actual  experiment  then  began  in  which  the  students  were 
exposed  to  the  ARPANET  and  its  command  and  control  research  technologies 
for  a  period  of  eleven  weeks  in  a  course  called  "Han-machine 
Interaction",  which  met  three  hours  per  week.  During  class,  video 
monitors  were  installed  around  the  room  so  the  students  could  see  the 
professor  actually  working  on  a  real-time,  interactive  basis  with  the 
ARPANET  and  various  computers  around  the  country.  Students  were  given 
specific  assignments  which  they  would  then  work  on  at  home  on  their  own 
computer  terminals.  Since  each  user  on  the  ARPANET  has  an  Individual 
electronic  "mail-box",  students  transmitted  their  finished  assignments 
to  the  professor's  mail-box  at  anytime  they  wished.  In  addition,  the 
professor  also  worked  on  the  NET  three  nights  a  week  from  home.  Because 
the  ARPANET  provides  a  teleconferencing  feature  between  users,  when  a 
student  had  a  question,  he/ she  could  simply  link  with  the  professor  and 
receive  Immediate  feedback  on  any  question.  Furthermore,  if  the 
professor  noted  several  students  with  the  same  questions  or  problems,  he 
would  have  them  all  link  together  and  then  go  to  the  particular  host 
computer  around  the  country  where  the  students  were  having  questions. 
He  would  run  through  a  short  demonstration  to  illustrate  t  e  answers  to 
their  questions,  and  during  this  time,  each  of  the  students'  terminals 
would  be  printing  out  an  exact  copy  of  what  the  professor  was  doing. 

Some  of  the  specific  research  products  and  some  of  their  human 
engineering  features  which  the  students  were  exposed  to  are  listed 
below: 
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1)  The  ARPANET  and  some  of  its  general  features.  A  small 
pamphlet  showing  how  to  access  the  ARPANET  and  use  one  of  its  mail 
handling  systems  was  given  to  each  student.  The  pamphlet  showed  the  user 
how  to  access  the  network  and  then  how  to  go  to  his  own  host  computer  on 
the  network.  The  pamphlet  explained  in  simple  terms  what  was  happening 
and  the  user  entered  given  commands  through  his  terminal  and  saw 
comparable  results  coming  back  from  the  system  as  those  shown  in  the 
pamphlet.  The  ARPANET  protocol  makes  extensive  use  of  an  escape 
character  wherein  the  user  can  start  to  enter  (log  in)  to  the  system  and 
by  typing  one  or  more  escape  characters,  the  system  knows  who  is  logging 
in  and  therefore  automatically  fills  in  account  data,  etc.,  for  the 
user. 

(NOTE:  The  human  engineering  features  and  user  designed  Interfaces  of 
the  ARPANET  were  quickly  re-emphasized  to  the  authors  after  they  had 
handed  out  the  terminals  and  the  small  introductory  pamphlet  to  the 
students  several  days  before  class  began.  Within  thirty  minutes  after 
they  left  one  of  the  authors'  offices  with  their  terminals,  the  authors 
were  also  logged  in  to  the  system  and  noticed  students  starting  to  log 
in  and  use  the  mail  handling  system.  That  was  impressive  considering 
that  the  host  computer  they  were  using  was  some  400  miles  away  and  they 
had  received  no  instruction.) 

2)  A  message  handling  system  (MSG)  -  which  basically  transmits 
pages  of  information  from  one  user  to  other  users'  electronic  mailboxes 
in  the  form  of  messages  or  letters.  Particular  features  which  make  this 
system  especially  useful  are  its  flexibility  and  its  automatic 
recognition  mode.  If  one  wants  to  send  a  message,  one  simply  types  an 
"s"  for  the  command  SEND  and  the  system  automatically  fills  out  the  word 
and  asks  for  the  addressee  and  the  host  computer  name  on  the  network.  In 
addition,  it  asks  for  addresses  of  people  you  may  wish  to  send  "carbon 
copies”.  The  pressing  of  one  key,  after  the  message  is  composed,  starts 
the  mailing  system  which  sends  the  message  to  all  addressees  specified 
and  the  system  shows  the  status  as  OK  when  each  user  has  been  sent  the 
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message.  If  a  communication  link  is  down,  the  message  will  be  listed  as 
queued,  so  the  user  knows  its  status.  Other  features  of  the  mail  system 
require  only  one  key  to  be  typed  to  invoke  them  such  as  "d"  for  deleting 
a  message,  "f"  for  forwarding  a  message  to  someone  from  someone  else, 
"a”  for  answering  a  message,  "u"  for  undeleting  a  message  which  the  user 
may  have  marked  for  deletion  by  mistake,  etc.  All  messages  in  the  user's 
mailbox  are  numbered,  so  to  answer  message  23,  the  user  simply  types 
a23 

and  the  system  fills  in  the  addressee  because  it  knows  whose  message  the 
user  is  answering. 

3)  A  text  preparation  and  editing  system  (XED)  -  which  is  used 
for  preparing  pages  of  text  for  longer  letters,  reports,  etc.  The 
system  makes  use  of  automatic  recognition  by  letting  the  user  type  "a" 
for  example,  and  the  system  fills  out  the  word  "append"  meaning  the  user 
wants  to  add  more  text,  etc.  By  typing  one  key,  the  user  can  also 
format  his  text  into  neat  pages  with  right  and  left  margins  aligned.  If 
the  user  wishes  to  save  the  text,  he  simply  types  "w"  and  the  system 
fills  in  "write  to  output  file"  where  the  user  fills  in  the  file  name. 

k)  Multi-host  computer  networking  (TELNET)  -  a  system  for  going 
from  one's  host  computer  to  another  on  the  network,  obtaining  some 
Information,  and  subsequently  returning  to  one's  own  host.  For  example, 
one  may  be  working  at  his  own  host  computer  and  decide  he  needs 
information  from  another  computer,  thousands  of  miles  away.  Instead  of 
having  to  terminate  operations  at  one's  own  host,  go  log  in  at  the  other 
to  get  the  desired  information  and  log  out  and  then  log  back  into  one's 
own  host  again,  this  system  allows  one  to  type  two  words  and  a 
connection  is  made  to  the  remote  host  from  the  user's  host  computer. 
The  user  can  get  the  desired  information  and  return  to  his  own  host  in 
exactly  the  same  place  he  left  it.  The  speed  and  procedure  with  which 
one  can  interconnect  several  computers  is  incredibly  easy  to  the  naive 
computer  user. 


5)  File  transfer  between  host  computers  (FTP)-  a  system  for 
allowing  one  user  to  retrieve  or  send  files  of  information  from  or  to 
another  user.  For  example,  one  user  wants  a  50  page  computer  program 
which  someone  else  has  at  a  host  computer  thousands  of  miles  away.  By 
typing  three  words,  a  connection  is  made  from  one  host  to  the  other  and 
the  50  page  computer  program  at  location  X  can  then  be  transferred  to 
location  Y  in  seconds.  The  user  at  location  Y  can  now  execute  that 
program  immediately  on  his  host  computer. 

6)  Search  and  rescue  demonstration  (SAR)  -  a  program  which 
allows  users  to  specify  a  disaster  site  and  subsequently  initiate  search 
and  rescue  operations.  With  mostly  YES  and  NO  responses,  plus  some 
numbers  entered  to  specify  distances,  the  user  actually  sees  automatic 
connections  being  made  around  the  computer  network  to  notify  specific 
people  of  the  simulated  disaster,  to  obtain  weather  data  In  the  vicinity 
of  the  disaster  from  other  computers,  and  to  determine  which  ships  might 
be  in  the  area  to  help  in  this  search  and  rescue  on  the  high  seas. 

7)  Inquiry  programs  -  to  determine  what  ships  are  at  sea,  where 
they  are,  what  they  have  onboard,  etc.  Two  inquiry  systems  were 
actually  used.  One  was  a  "natural  language"  query  system  which 
approaches  the  ultimate  in  man-machine  interface  by  some  standards.  It 
allowed  the  user  to  ask  questions  in  his  own  words  and  the  user  could 
then  see  the  program  trying  to  parse  or  understand  his  question.  If 
unsuccessful,  the  program  replied  with  reasons  why  it  didn't  understand. 
If  successful,  the  user  would  see  the  program  connect  to  other 
distributed  data  base*  around  the  country  to  obtain  the  answers.  In 
many  cases,  this  system  also  corrected  user  typing  errors  in  input. 
From  a  man-machine  interface  viewpoint,  this  system  is  very  flexible  in 
that  one  user  can  ask  "Where  is  the  John  F.  Kennedy?"  and  another  user 
can  ask  "What  is  the  position  of  the  JFK?"  and  both  users  will  get  the 
same  answer. 

The  students  also  used  another  query  system  which  was  somewhat 
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in  contrast  to  the  natural  language  system  and  built  on  a  different 
philosophy  of  the  best  man-machine  interface.  In  this  system,  a  tree 
structure  is  used  in  which  a  user  learns  certain  key  words  and  tree 
branches.  The  system  then  uses  automatic  recognition  to  help  fill  in 
the  user's  request.  For  example,  a  user  might  wish  to  say, 

"Show  Russian  submarines  within  350  nautical  miles  of  Honolulu".  The 
user  would  actually  type  the  following  letters:  S  R  SU  W  350  N  Honolulu 
and  the  system  would  fill  in  the  words  to  make  the  sentence  appear  on 
the  terminal  as  written  above  in  the  quotes.  With  a  few  hours  training, 
the  user  can  basically  use  this  system  in  a  sort  of  short  hand  way  by 
just  entering  the  first  letter  of  words  and  the  system  automatically 
types  out  the  rest  of  the  words  plus  some  others,  i.e.  if  the  user  types 
N,  the  system  prints 

"Nautical  miles  of...". 

The  above  descriptions  are  brief  but  intended  to  show  some  of 
the  types  of  research  products  the  subjects  were  exposed  to,  and  to  give 
an  idea  of  some  of  the  man-machine  interface  characteristics  which  had 
been  designed  into  the  original  product. 

It  is  important  to  note  here  that  the  authors,  in  using  the 
above  systems  and  research  products,  had  attempted  to  take  these  systems 
to  the  user,  who  as  a  naive  computer  expert,  could  exercise  and  run  them 
successfully  with  few  problems.  The  systems  introduced  were  realistic 
products  which  the  subjeots  could  relate  with,  versus  typical 
Introductions  to  computers  in  which  a  certain  example  is  used  and  the 
user  is  required  to  imagine  and  translate  this  into  how  it  could  be  used 
in  his/her  dally  tasks.  In  addition,  these  were  systems  which  had  been 
designed  initially  with  the  user  in  mind,  versus  many  systems  designed 
to  do  a  task,  and  then  the  user  must  find  out  all  the  small  details  to 
make  them  run.  For  example,  in  almost  every  system  described  above, 
typing  HELP  or  a  question  mark  will  provide  general  assistance  on  the 
system  or  answer  a  user's  specific  questions. 


RESULTS 


The  same  questionnaire  that  was  administered  before  the 
experiment  was  given  at  the  conclusion  of  the  course  with  all  subjects 
responding.  The  purpose  was  to  determine  if  the  exposure  to  these  user 
designed  systems  had  positively  influenced  the  subjects'  perceptions  and 
if  so,  in  what  direction. 

The  responses  to  the  questions  had  been  on  a  scale  from  A  (None 
or  very  little)  to  E  (A  great  deal)  and  were  therefore  considered  an 
ordinal  measurement.  Since  the  magnitude  of  the  change  in  responses  was 
not  considered  meaningful  (a  change  in  response  from  C  to  E  was  not 
necessarily  equivalent  to  a  change  from  B  to  D),  the  authors  evaluated 
the  subjects'  responses  to  each  question  with  the  Sign  Test  (Siegel, 
1956).  This  test  basically  showed  if  a  significant  change  had  occurred 
in  the  group's  opinion  during  the  experiment.  A  significance  level  of 
. 10  was  used  since  the  authors  were  looking  for  general  trends  in  the 
subject  group's  opinion. 

Subjects  showed  a  significant,  positive  change  in  response  to 
the  question  "To  what  extent  are  you  interested  in  personally  working 
with  computer  systems?".  After  the  experiment,  they  were  more 
Interested  in  personally  working  with  such  systems.  The  reader  will 
remember  that  before  the  experiment  started,  the  subjects  had  no  desire 
to  really  get  in  and  work  with  such  systems,  even  though  they  had 
previously  received  a  course  in  FORTRAN  and  a  course  in  elementary 
graphics. 

The  subjects  also  showed  a  significant  positive  increase  in 
their  perceptions  of  how  they  could  have  used  computers  in  their  past 
Jobs  and  in  future  jobs.  After  the  experiment  they  felt  more  strongly 
that  they  could  have  effectively  used  a  computer  in  past  jobs,  and 
further  they  now  felt  more  strongly  that  it  would  be  practical  to  expect 
future  military  commanders  to  use  personal  computer  terminals.  On  a 
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related  question,  they  also  had  a  oueh  more  significant  positive 
attitude  after  the  experiaent  that  they  would  prefer  to  use  interactive 
terminals  themselves  rather  than  having  a  subordinate  perform  this 
function . 

When  asked  "To  what  extent  do  you  view  the  utility  or  value  of 
computer  systems?”,  the  subjects  showed  no  change  of  opinion.  This  was 
rather  perplexing  until  the  group  of  subjects  was  interviewed  a  few 
weeks  later  and  they  replied  "We  always  thought  computer  systems  were 
valuable  and  we  still  do... but  now  we  want  to  be  involved  with  them 
whereas  before  we  really  didn't." 

Subjects  also  showed  a  significant  increased  perception  of  the 
benefit  of  computerized  decision  aiding  systems.  This  was  probably 
because  they  had  been  able  to  experience  such  systems  and  gain 
confidence  in  the  ease  with  which  the  systems  could  be  used.  They  also 
probably  gained  oonfldence  in  the  reliability  of  the  systems. 

There  was  only  one  question  in  which  the  subjects'  perceptions 
significantly  decreased  on  the  rating  scale.  The  subjects  now  felt  it 
was  less  likely  for  them  to  possibly  misunderstand  someone  else's 
written  message.  Although  the  experiment  did  not  include  any 
information  on  how  to  oompose  or  understand  a  written  message,  the 
subjects  may  have  thought  they  understood  messages  better  because  on  the 
few  times  when  they  did  have  a  question,  they  could  often  link  up  with 
the  message  sender,  wherever  that  person  was  on  the  ARPANET,  and  get  an 
Immediate  clarification  of  any  question  they  had  about  the  sender's 
message. 

After  the  experiment,  the  subjects  also  showed  a  significant 
Increase  in  their  responses  to  the  question  "To  what  extent  do  you  find 
it  easy  to  quickly  send  the  same  information  to  many  different  people". 
Indicating  it  was  now  easier  was  probably  due  to  the  fact  that  in 
previous  experiences,  they  used  the  typical  business  approach  of  having 
information  typed,  then  copies  made  in  some  fashion,  and  finally 
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envelopes  addressed  and  sent  through  normal  mall  channels.  With  their 
experience  on  the  ARPANET,  the  subjects  had  been  able  to  prepare 
information  and  electronically  send  it  within  minutes  to  multiple 
addressees  simultaneously,  anywhere  on  the  network,  thousands  of  miles 
apart. 


The  other  questions  dealt  with  more  specific  areas  of  interest 
and  are  not  of  importance  here.  For  example,  one  question  asked  "To 
what  extent  are  you  familiar  with  artificial  Intelligence  applications?" 
and  obviously  most  showed  an  increased  familiarity  since  they  were 
Introduced  to  some  artificial  intelligence  applications  during  the 
experiment.  One  similar  type  of  detailed  question  was  revealing  from  a 
man-machine  Interface  viewpoint.  That  question  asked  "To  what  extent 
does  the  ability  to  type  enhance  your  ability  to  communicate?".  After 
the  experiment,  the  subjects  showed  a  significant,  positive  increase  in 
their  responses,  indicating  that  they  now  felt  the  ability  to  type  was 
even  more  important.  This  has  significant  implications  for  the  future 
if  the  current  trend  continues  toward  the  use  of  more  computer  systems 
in  our  daily  lives.  Current  trends  are  toward  more  interactive  use  via 
computer  terminal,  and  if  other  modes  of  data  entry  are  not  developed, 
the  acquistition  of  typing  skills  in  ones's  earlier  schooling  years  will 
be  extremely  valuable  in  later  years. 

Typically,  after  students  in  a  computer  type  class  have  finished 
their  course,  their  use  of  computers  and  computer  systems  will  drop  to 
practically  zero,  if  in  the  following  weeks  and  months,  they  are  not 
taking  more  computer  courses  or  working  on  thesis  research.  A  further 
verification  that  the  subjects  in  this  experiment  had  been  motivated  or 
"turned  on”  by  the  experimental  experience  they  received,  is  indicated 
in  accounting  data  for  the  following  two  months  after  the  experiment  had 
ended.  Although  these  subjects  were  taking  no  courses  requiring  use  of 
a  computer  and  none  of  them  were  working  on  thesis  research,  they  have 
averaged  8  hours  per  week  per  subject  using  the  facilities  and  research 
products  provided  them  during  the  experiment  and  this  has  continued  for 
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two  months  up  to  the  time  of  this  writing. 

Another  interesting  aspect  revealed  in  the  subjects'  responses 
was  a  significant  change  in  their  attitude  toward  deleting  files  of 
information.  After  the  experiment,  they  felt  much  more  comfortable  in 
creating  and  destroying  files  of  information.  This  is  probably  due  to 
their  previous  experiences  in  which  most  computer  system  users  are  told 
very  specifically  how  to  delete  information,  because  it  will  be  gone 
forever  if  one  deletes  it.  The  ARPANET  has  been  designed  however,  such 
that  a  user  may  delete  information,  but  if  the  user  realizes  at  anytime 
before  he  logs  out,  that  he  made  a  mistake  and  really  needs  the  file  of 
information,  he  oan  simply  undelete  it  and  he  will  have  it  restored  in 
its  original  form.  This  is  a  nice  feature  for  the  user  and  certainly 
had  an  influence  on  the  subjects  during  this  experiment. 

A  purely  subjective  observation  by  the  authors  also  revealed 
that  subject  interest  levels  started  out  very  high  and  remained  so 
throughout  the  experiment.  This  observation  is  based  not  only  on 
accounting  data,  but  on  the  enthusiasm  and  excitement  exhibited  by  the 
students  in  daily  contacts  with  them. 

At  the  end  of  the  experiment,  the  subjects  were  also  asked  to 
indicate  which  of  the  query  systems  they  preferred  . . .  the  natural 
language  query  system  or  the  less  natural,  tree  structured,  query  system 
described  earlier.  Their  preference  was  to  be  based  on  the  man-machine 
interaction  with  the  two  systems  and  on  how  they  could  formulate 
queries.  Their  preference  was  not  to  be  based  on  what  type  of 
information  they  could  obtain  from  the  system.  The  subjects  were 
divided  into  two  groups  called  OPERATORS  and  SUPPORT.  OPERATORS  means 
airplane  pilots,  ship  drivers,  etc.  SUPPORT  refers  to  individuals  in 
communications,  intelligence,  etc.,  whose  main  function  is  in  support  of 
the  operational  areas.  This  is  analagous  to  the  staff  function  in  any 
large  organization.  Six  OPERATORS  chose  the  natural  language  system  and 
two  OPERATORS  chose  the  tree  structured  query  system.  However,  seven  of 


the  SUPPORT  group  chose  the  tree  structured  query  system  and  two  of  the 
SUPPORT  group  chose  the  natural  language  system.  Arranging  this  data  in 
a  2  X  2  matrix,  the  Fisher  Exact  Probability  Test  (Siegel,  1956)  was 
applied  and  showed  a  significant  difference  (p  <  .05)  in  the  two  groups' 
preferences.  The  OPERATORS  definitely  preferred  the  natural  language 
type  query  system,  and  in  reality,  they  would  usually  be  flying  planes, 
etc.,  i.e.  performing  the  "command"  functions,  and  only  occasionally 
have  need  to  go  into  a  command  center  and  enter  their  own  query  into  a 
system.  As  such  they  would  prefer  to  not  have  to  remember  details  of  a 
system  *yit  rather  enter  their  own  "natural"  query.  The  SUPPORT  group 
definitely  preferred  the  tree  structured  query  system  which  allowed  them 
to  use  a  "shorthand"  version  of  query  entry,  and  thus  enter  queries  much 
faster.  In  reality,  the  SUPPORT  group  would  use  such  systems  on  a  daily 
basis.  This  revealing  aspect  of  query  system  preference  indicates  that 
although  the  current  trend  today  is  toward  development  of  natural 
language  systems  in  many  fields,  one  should  carefully  examine  the  daily 
user  to  determine  if  the  additional  cost  of  a  natural  language  system  is 
really  worth  the  amount  of  use  the  system  will  receive,  or  would  a  less 
natural  language  system  in  fact  serve  the  users'  purposes  better? 

CONCLUSIONS 

The  transfer  of  technology  from  designer  to  user  is  often  a 

difficult  process  and  many  times  the  transfer  does  not  occur  because  the 

technology  designer  did  not  consider  how  the  technology  would  be  used  by 

the  end  user.  In  this  experiment,  a  broad  range  of  research  technology 

products  were  presented  to  users.  However,  the  research  technology 

products  were  all  ones  that  had  considered  human  factors  in  their 

initial  design  stages,  and  as  a  result,  subjects  found  the  products  easy 

to  use.  Consequently,  user  perceptions  of  computer  systems  and  computer 

software  projects  were  greatly  enhanced,  and  in  addition,  user 
and 

motivations^  interest  levels  in  working  with  these  systems  were  greatly 
elevated  and  have  remained  so. 


Two  comments  might  be  made  in  closing.  Since  the  subjects  in 
this  experiment  were  professional  military  officers,  with  current 
operational  experience,  they  provided  timely,  real-world  comments  to 
designers  on  how  to  improve  their  products,  with  such  improvements  being 
made  immediately  in  several  cases.  Finally,  a  side  benefit  derived  from 
providing  each  subject  with  his  own  portable  terminal  at  home,  was  that 
the  subject  could  work  at  home  at  night  and  still  be  with  his  family,  as 
opposed  to  the  more  typical  situation  in  which  the  student  SDends  many 
nights  at  the  computer  center  and  the  family  begins  to  wonder  if  he 
doesn't  really  live  at  the  computer  center. 

An  overview  of  this  experiment  strongly  suggests  that  a  viable 
method  of  technology  transfer  may  be  through  special  programs  which 
"invite"  the  subject  or  user  to  become  familiar  with  the  technology  and 
evaluate  it.  It  might  also  be  suggested  that  the  human  interface  with 
automated  systems,  expecially  decision  support  systems,  can  be 
substantially  enhanoed  through  a  properly  designed  exposure  to  the 
technology  and  an  opportunity  to  use  it  without  fear  of  failure. 


NOTE:  Follow  on  observations  of  the  2nd  and  3rd  C3  classes  also 

indicated  the  same  trends  and  results  as  found_in  this  study. 
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